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ABSTRACT 

A multi- variable curricular model is applied to a 
total baccalaureate curriculum revision. The objectives of the 
evaluation, methods, and sources are given. The model can he used to 
examine how much additional power in decision making was gained by 
each set of variables at the model. (Author) 
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A MODEL OF CURRICULUM EVALUATION 
APPLIED TO A UNIVERSITY BACCALAUREATE PROGRAM 
Vivian C. Wolf and Cecilia 11, Smith 
University of Washington 

Curriculum development and evaluation reminds uo of group sky diving. 
Many efforts must be made to decide on the appropriate target (goals) 
and you may get the airplane (grant) to help you get in a position to 
begin (take the big leap). There are a multitude of forces that will 
affect your progress toward the goals or target. You do have some limited 
number of strings you can pull to control your situation, but there may 
be forces beyond your control. You may find your group may never 
get together and all land off target, or if someone misjudges the time 
sequences or actions of others, disaster may occur. Curriculum <I«velop- 
ment and evaluation today is frequently accelerated to a speed ar which 
decisions come very'rapidly forcing continual skilled fast action and 
reassessment of priorities. When a large number of persons are involved 
in an project, the communication and coordination problems increase as 
do the variables that should be considered. 

The faculty of the University of Washington School of Nursing felt 
the need for a revised curriculum to prepare students for extended roles 
in nursing and the delivery of care in emerging health care systems. 
The last major revision of the baccalaureate nursing curriculum at the 
University o£ Washington was initiated in 1952 and implemented in 1959. 
Subsequently, the resultant changes in perceptions of the role of the 
nurse practitioners, the delivery of health care, the expansion of knowledg 
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and concerns about the adequacy of the current curriculum in meeting 
the desired educational goals, led the faculty to the decision that a 
curriculum grant was needed to bring about a curriculum revision, A 
grant proposal to support th^ curricMlu:a revision, written in September 
1970, was funded June 1, 1971. 

This paper will present the beginning steps taken by the staff of 
the curriculum grant to develop and evaluate the school's proposed 
curriculum changes. Because it is felt that there are some fundamental 
issues in relation to curriculum evaluation which are the subject of 
much controversy, ve believe that the evaluator must make some basic 
theoretical decisions before proceeding with his evaluation plan. For 
this reason a brief review of some of these issues will be addressed 
before presenting our activities in relation to evaluation. Because we 
are presenting an overview of a plan for evaluation of the change of a 
total baccalaureate program, time and Space does not permit a large 
amount of detail on specific aspects. We hope to present detailed com- 
ponents in later papers. This paper will be organized into four major 
parts: Some Major Theoretical Issues in Curriculum Evaluation and Selec 
tion of an Evaluation Model; Factors Within Our Situation that Modify 
the Scope and Rigor of the Evaluation; Goals, Progress and Problems 
in Relation to the Model and Summary. 

Some Major Theoretical Issues in Curriculum 
Evaluation and Selection of an Evaluation Model 
Definition of Evaluation 

There are numerous definitions of evaluation. Three examples 
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will be cited. First, one of the best analyses dona on the inadequacies 

of the various definitions of evaluation is presented in the book written 

by the Phi Delta Kappa National Study Coinniittee on Evaluation (PDK Study 

Committee) chaired by Daniel Stuff lebeam (1971). The committee, after 

deliberating on the numerous inadequacies, developed a new definition of 

educational evaluation: "Educational evaluation is the (process) of 

(delineating), (obtaining), and (providing) (useful) (information) for 

(judging) (decision alternatives)* (1971, p. 40). Second, Robert Rippey 

in an article which asks the question "Can Evaluation Research Be Integral 

to Innovation?" (1972, p. 45-58) cites many of the problems with the 

present conceptions of evaluation and concludes, 

At the moment, there seems to be no evidence that evaluation, although 
the law of the land, contributes anything to educational practice 
other than headaches for the researcher, threats for the innovators, 
and depressing test for journal iscues devoted to evaluation. 
(1972, p. 46) 

Rippey feels that the primary source of resistance to change is not 
economic or technical but is due rather to the effect change has on the 
roles of school personnel. For this reason he advocates a model of 
transactional evaluation which involves not only the protagonists and 
designers of the innovation but also a representative sample of persons 
likely to be affected adversely or disturbed by the consequences of the 
change. This transactional analysis is directed not only at program 
improvements but also at the analysis of organizational dysfunction 
as a result of change due to the threats to role stability. The trans- 
actional evaluation is carried out by a series of questionnaires admin- 
istered to various involved groups and a series of decisions based on this 
information is then followed by more evaluation. 
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A third definition of evaluation is stated by Glass and Worthen: 



"Evaluation is the determination of the worth of a thing. It is 
the process of obtaining information for judging the worth of an 
educational program, product, procedure, or educationa-L-oblective 
or the potential utility of alternative approaches designed to attain 
specified objectives." 

Curriculum evaluators must make a decision about what definition of 
evaluation they will choose to endorse. We "feel that Rippey'", trans- 
actional evaluation and the definition of evaluation by Glass and Worthen 
could be seen as being subsumed under the PDK Staff definition. Riopey's 
transactional analysis is only one means of gathering data for jiriging 
alternatives, and one type of decision alternatives may be about the 
"worth of a thing." Because we view the PDK definition as broader and 
reflecting our view of what we are trying to accomplish, we wi?l accept 
it as the definition to guide our evaluation efforts. 

The Issue of Research Versus Evaluation 

There is confusion in the minds of many as to what is research and 
what is evaluation. Glass and Worthen endecvor to differentiate carefully 
the similarities and the differences between research and evaluation. 
Readers arc referred to their excellent work for elaboration of these points. 
Two of their statements seem especially helffful to the curriculum specialist 




working on evaluation problems. 

There is considerable confusion among investigators ir education about 
the extent to which evaluators should explain ("understand") the 
phenomena they evaluate. A fully proper and useful evaluation can 
be conducted without producing an explanation oc why a product 
or program being evaluated is good or bad or of how it operates to 
produce its effects. It is fortunate that this is so, since equation 
in education is so needed and credible explanations of educational 
phenomena arc so rare. 

When discussing the kinds of controls that can be used they latter 
indicate: . . . 
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It is enough for the cvaluator to know that something attendant 
upon the installation of Curriculum A (and not an extraneous, 
"uncontrolled" influence unrelated to the curriculum) is responsible 
for the valued outcome. To give a more definite ansv7er about what 
that something is would carry evaluation into analytical research. 
Analytical research on the non-generalizable phenomena of evalua- 
tion is seldom worth the expense (Glass U Worthen, 1972, p. 160). 

One gains the impression from reading the Glass and Worthen article 

that you either decide you are going to do research or you decide you are 

going to do evaluation and then recognize and utilize the differences. 

It is our view that this impression is given because the major purpose of 

the article is to heighten the contrast of research versus evaluation. 

We find this helpful but are attempting to examine the major parts of 

the overall evaluation program to see if parts of that program are able 

to be and should be addressed as research. Therefore, we do not see our 

work as all evaluation or all research but rather we secLjthe opportunity 

for some research within the overall evaluation context. 

The Model of Evaluation 

Numerous models of educational evaluation have been proposed. It is 
necessary to develop or select a model which best represents the course 
of action that the curriculum evaluators feel will farther their evaluation. 
After a review of some of these models and thinking about how we planned 
to approach our evaluation, we have decided *;o use the PDK EvaluuLion 
Model as represented by William Gephart (1972, pp. 118-119). A schematic 
representation of this model is shown in Fig. 1. Gephart explains the 
parts of the model in the following manner; 

The evaluation model proposed has multiple levels. The first level is 
a continuous effort called "context evaluation," It serves planning 
decisions by determining congruence between the objectives accepted 
by the educational system and its actual accomplishments, as well 
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as identifying those contingencies in the institution's environment 
that may detail additional objectives to be considered. At a second 
level is "input evaluation." It is undertaken when a decision to 
change is reached and it provides inform tiori that weighs the alter- 
native ways of achieving the desired change. At the thirJ level is 
"process and product evaluation." These coma into being when 
an input alternative is selected for which insufficient information 
exists regarding its procedures, its effects, or both. This level 
of evaluation provides information on what events occurred and • 
what they produced. The information in turn serves decision alter- 
natives related to events that follow a field test of an approach 
to meeting a new objective. Such alternatives include : terminate 
the frial, recycle the trial with some modification and implement 
the procedures of the trial in the overall syst?m (Gephart, p. 130). 

We feel that the PDK definition of evaluation and Gephart 1 s model 

raise some fundamental questions* If you say that "educational evaluation 

is the process of delineating, obtaining and providing useful information 

for judging decision alternatives," then how doe's this differ £-om the 



process of curriculum development or do we have a n c r v marriage? Kas 
there been a secret marriage for some years that we have just discovered? 
When this new definition is considered in view of TyJer's (1949) flasic 
Principles of Curriculum and Instruction several questions are raised. Was 
Tyler engaged in context evaluation when he looked at the society, the 
learners and the subject matter specialists? When he compares the objectives 
with the philosophy and learning theory was he using input evaluation as 
described in the PDK model? Were the concepts of continuity, sequence, 
integration and organizing principles criteria utilized in input evaluation? 
Did Tyler's section on evaluation in the last part of his book relate to 
process and product evaluation that feed data back in*o the context 
evaluation? All of this becomes even less distinct in our opinion if you 
use a definition of "the practical" for curriculum such as Schwab's (1970) 
which emphasizes decision making in the curricular process. One might 
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ask what these questions have to do with an evaluation program? If it is 
agreed that these definitions now bring about a high interaction or 
overlapping between the process of curriculum development and curriculum 
evaluation, then this interaction must be highlighted or some persons will 
say you are not describing your evaluation program but your curricuLar 
process. We view the process of curriculum development as a decision 
making process and will not in this paper attempt to clarify where we see 
the curriculum development process as distinct from curriculum evaluation 
as defined by the PDK Study Committee. The similarities and differences 
may be the subject for another session or paper. 

The Range of Data that Should be Included in Evaluation 

The list of things that one can study in curriculum evaluation varies 

considerably with the writers. A few of these suggestions are: 

Student characteristics - values, perceptions, interests, achieve- 
ments, attitude, aptitude, social background, intelligence, education- 
al background, creativity and other process skills, job and activity 
experience, performance after graduation. 

Faculty chara cteristics - values, educational background, perceptions 
of the administration or their profession or the school, faculty 
teaching skills and styles, work experience contribution to the pro- 
fession and creativity. 

Environment - space, lighting, multi-media equipment, offices, color, 
use of space, location, the social conditions that affect campus 
unrest or funding. 
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Curricular walc; riol_s " the objectives, philosophy, theory of learning, 
course outlines, software. 

Economic consi der ations - cost of materials, mu^ ti-media , cost of 
facilities, cost of time spent, faculty time spent. 

Process of curriculum chang e - communication networks, decision makers, 
sequence. 

As indicated above, the list of variables that one could consider and 
that have been judged to affect the performance of a student or groups of 
students in a given curriculum is extensive. Most current writings on 
curriculum evaluation advocate that you study a wider range of factors. 
In the end the evaluation program is usually a choi<* among many possibilities 
which is narrowed by time, economics, knowledge, values, skills and 
personnel as well as relevance to the problem. 

The number and types of personnel that should be included in the e valuation 

There are at least two parts to this issue - who should be studied and 
who should do the studying. At one extreme there are those who advocate the 
examination of effects on nearly everyone who could possible be affected by 
the proposed change. This would include persons such a.", the parents, 
the students, faculty and administration. At the-dpposite extreme 
there are those who advocate studying just the learner. The second part 
of this issue concerns the composition of the evaluation team. Some ad- 
vocate as ideal an evaluation staff which would include a writer, designer, 
and from six to ten people or relay 'teams of outside evaluation specialists 
who compare their independently gathered notes. As individuals involved 
in curriculum development, we recognize t!*at some of these roles and 
functions would be highly desirable. We will try to remember to bring 
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these views into our next grant proposal as expert opinion on the personnel 
needed, but we are realistically not very optimistic about how much addi- 
tional money these ideas will enable us to obtain. 

Weiss in his introduction to the Curriculum Theory Net work Monograph 
on Evaluation states: "Much of the curriculum evaluation literature does 
not provide procedures that can be readily adopted or adapted nor does it 
offer concrete examples of actual evaluation studies ." (1972, p.l) Later 
in closing comments, he states: n At this stage in the work of curriculum 
evaluation, there appears to be more written about the models (of evaluation) 
than accounts of their use in educational practice. " (1972, p. 246) Perhaps 
when the difference between idealism and what most of us can achieve gets 
to be too great people either retreat from doing anything or refuse to share 
their realities because they are afraid ideal standards will be applied 
to their less than ideal situations. This leads us to the second major 
part of this paper. 

Factors Within Our Situation that 
Modify the Scope and Rigor of the Evaluation 
Limited grant funds 

Our grant funds are limited in time and amount. To assist approximately 
one hundred faculty to revise a total nursing baccalaureate program our grant 
has one full time Director, a nurse with a Ph.D. in Curriculum and Instruction, 
one Assistant Director with post-master's preparation in Educational Anthro- 
pology, one nurse with a B.A. who spends ten hours a week in advising 
and ten hours a week working on grant data gathering, one half-time student 

research assistant, and two secretaries. The Director also teaches a 3-5 

I 

credit curriculum course to approximately 100 graduate students each year. 
We are the developers and the evaluators. More personnel were written into 
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the grant than were funded* The realities of the present age and our 
consultant visits to other schools convince us that we are in a better 
position than" many. Even though we feel we have worked rapidly to get 
our philosophy, objectives and all course outlines developed by the second 
year of our grant, we still find that no class completing the new .curriculum 
will graduate before our funds for this grant run ou'J. We can only hope 
that by establishing a good evaluation system we will be able to secure 
^another grant to continue this evaluation. 

Time factors in the actual curriculum situation 

It took from 1969 until the Fall of 1972 for our faculty to delineate 
and agree upon their terminal objectives, philosophy, curricular model, and 
course outlines. Once all of this was done it has to be reviewed by: 

The Board of Health Sciences, 

The University Curriculum Subcommittee on rtew and Revised Curriculum, 

The University Curriculum Committee, and 

The State Board of Nursing, 
Providing all the necessary materials to all these groups and meeting with 
them took about four months even though not one group recommended any 
changes. The State Board of Nursing requires that all curricula that 
are new or revised must be approved by them six months prior to the admission 
of a first class and before any publicity can be released. Wc submitted 
our bulletin to the printers in January, and with good fortune it will be 
in the hands of high school students by March, so our first class in the 
new program can be admitted in the Fall of '73 and our first nursing content 
will be offered in Winter 1975. 

— - And Noah though 40 days and nights were long I 



Wolf 11 

If a curriculum change extends ever a longer period of time, the 
curriculum materials may become outdated before, the revision is completed. 
This is especially true if the profession is changing rapidly as is 
nursing. Good formative evaluation will reduce this danger. The longer 
the period of time the curriculum work is extended, the more ex-^^eous 
variables can affect the changes and evaluation* Therefoi . is helpful 
to get the curriculum changed rapidly once it is known what is desired. 

A short period of time for curriculum development and evaluation may 
not be an ideal situation either. When high production demands exist to 
be met in a short period of time with a limited staff, the staff may 
experience continual pressure to produce or refine curriculnr materials 
and coordinate curriculum production. At the same time they are supposed 
to be developing and incorporating a well though: out evaluation plan 
and selecting or developing baseline tests. 

School and state budgets 

In the opinion of all of our administrators, school and state budgets 
do not allow us enough money to run the current and proposed programs 
along side one another. Goodbye, hopes of .experimental designs. 

Consultation quests 

Consultation funds in these grants are limited. If your consultants 
travel from some distance you can probably have two visits a year of two 
or three days. Or you can try to obtain excellent local assistance from 
consultants who can give a higher input of time for the same total expen- 
diture of funds. Our grant allows us to pay about $100 per day which 
eliminates the possibility of engaging some consultants we might feel would 
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be beneficial to our grant. There are very few consultants in nursing 
who kno' nursing, curriculum development, and curriculum evaluation. Non- 
nurse consultar cogently find it difficult to deal with the professional 
content of nursing which may be divorced from their subject matter background. 

Goals, Progress and Problems in Relation to the Model 
The overall aim of the grant proposal was to promote a major revision 
in the curriculum on the basis of an evaluative analysis of the current 
program in order to more adequately prepare nursing students to meet 
health needs of society. Five specific grant aims were identified: 1) To 
state the needs our curriculum should try to meet; 2) to formulate a con- 
ceptual framework for curricular change; 3) to propose a revised curricular 
plan for the baccalaureate program in nursing; 4) to implement the proposed 
curricular plans, and 5) to evaluate the contribution of the new curriculum. 
T^rse aims extend over time and include evaluation at context, input, process, 
and product levels. The first .aim, to state the needs our curriculum should 
try to meet, addressed itself to the data of context evaluation needed for 
decisioii making and progression into input evaluation. 

Context evaluation 

Context evaluation, as described by the PDK Study Committee encompasses 
two major foci: 1) the forces within the environment indicating needs to be 
met, and 2) the problems blocking the fulfillment of these needs, both of 
which constitute the basic input necessary for the determination of the 
rationale for change (Stuff lebeam, 1971, *pp. 218-219). 

Process of decisio n maHng - Dedicated ;to the "democratic" process of 
decision making, the faculty adhered to a process whereby small groups 
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of task forces, student panels, interdisciplinary groups, individual inter- 
views and position paper presentations fed into workshops and ultimately 
total faculty discussion groups, providing input for decisions by total 
faculty numbering over 100, It soon became apparent that provision of 
freedom to individuals within the groups created problems of promoting 
optimal group process versus maintaining the rigors of effective evaluation. 
Obviously, decisions arrived at by such means were the subject of much 
discussion, and negotiation w£s often necessary. 

Initial overview of the rationale for change . As a basis for decisions 
regarding the overall rationale for change in the context evaluation, the 
faculty began preliminary steps of data collection in 1969. 

A school of nursing standing committee, the Faculty Staging Committee, 
served as an informing, coordinating and advisory working group in the 
initial planning for the curriculum revision. In order to identify the 
needs our curriculum should attempt to meet, a Task Force on Rationale for 
Curriculum Change was formed. A preliminary review of the literature in 
relation to these needs and the role of the nurse in meeting the health 
service needs of society was done by this task force. Thirteen general- 
izations were derived from supporting statements from the literature. 
These statements centered on health care, nursing, education, learning 
and students. It was the decision of the faculty in December 1969 to 
accept these statements as a basis for further discussion and a starting 
point for deliberation on philosophy statements. 

A Task Force on Long Range Goals reviewed literature under the headings: 
Social Change and Health Services, Trends in Nursing. Education, Functions 
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of the University, the Center for the Health Sciences and the School of 
Nursing, Long Range Goals for the School of Nursing and Goals fcr the Under- 
graduate Program, and made specific recommendations, -These recommendations 
formed the basis for decisions made by the faculty to develop innovative, 
ways for providing clinical experience which would be less expensive in 
the use of faculty time and more effective in providing competance in 
nursing skills. Deliberations provided additional input indicating the need 
to identify key concepts of nursing that influence the expected outcomes for 
graduates. The desirability of interdisciplinary courses, utilizing con- 
cepts of health care, was stressed and became the basis for later decisions 
to design interdisciplinary courses for inclusion in the revised curriculum. 

Health care delivery . Because health care dsiliveiry *as chaiging so 
rapidly, another Task Force on Health Cere Delivery Jn the Future was 
formed in 1970 to provide additional input for context evaluation. Other 
disciplines in the health sciences were invited to participate in developing 
a report and contributions were received from a physician, a nursing 
service director, a pharmacist and twelve nursing educators and students. 
This material was discussed by all the faculty at a workshop in Hay 1971. 
Changes in population needs, expanding roles of the nurse, and networks of 
health care centers were reviewed. Increased awareness on the part of the 
faculty in regard to these considerations formed the basis for later dis- 
cussions in regard to content of major areas of core in the revised curriculum. 

In April 1970 a Task Force on Social Action was formed. This group of 
faculty and students focused upon relevant social issues and the nurses' 
role in regard to social action. Based upon their definition of social 
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action and curricular implications, a decision was made by the faculty to include 
a terminal objective in reference to the graduate as an agent of change and 
promote and extend educational experiences allowing the student to broaden 
her leadership skills throughout the program. 

Because the extended role of the nurse was being rapidly modified, 
the faculty examined clinical specialisation , where it should occur and what 
components within the baccalaureate .curriculum would assist in the devel- 
opment of competent practitioners. As a result of these deliberations, 
the decision was made to provide for an intensive block of experience in 
selected special fields. Objectives of this component were to be carefully 
designed to allow for flexibility in anticipation of changing health 
care needs in the future* 

' Consumer views of heal th care needs and provisions. Because our 
rationale for curricular change indicated that the consumer was having 
more influence on the health care delivery system and because our approach 
to nursing has been consumer centered, a Task Force on the Consumer was 
formed in 1970 to study consumers 1 experience with nurses, the consumers* 
views of the future of health care delivery and the consumers' views of 
how nurses could be used differently. This task force interviewed eight 
groups of consumers from different socio-economic levels and put a quest- 
ionnaire in a local newspaper* Input from these endeavors provided context 
evaluation materials for further deliberation at a faculty workshop. Views 
of consumers that nurses were capable of additional responsibility, which 
they hesitated to assume, provided input which corresponded to views of 
students and faculty and led to decisions concerning content such as the 
intensive experience in a special area, in the revised curriculum. 
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E valuation of current curriculum plan . In addition to the input 
received, which indicated to our faculty some necessary directions to proceed 
in order to meet the needs of society and directions for nursing care providers 
in the future, the need was felt to evaluate the present curriculum plan in 
order to determine its effectiveness. To achieve this aim, a Task Force 
on Repetitions and Gaps was formed in October 1969. Because the task 
force members felt that nursing process was central to nursing education 
they decided to study the way in which the nursing process is taught and 
the variety of nursing care forms used. This study resulted in a com- 
parative analysis of nursing care forms used, decisions to provide for 
a strong emphasis on nursing process in the revised curriculum and definition 
of terms used in the terminal objectives, 

A student panel also presented their views of changes they would 
like to see included in the revised curriculum. Areas of content not 
previously stressed, such as alcoholism, drug abuse, first aid, emergency 
care, and research influenced subsequent decisions to incorporate areas of 
content in new course outlines. 

Several other sources of information were available to evaluate the 
current curriculum plan. Follow-up studies of the graduates of the classes 
of 1962 and 1963 were published in Nursing Research (Brandt, 1967, pp. 50-60.) 
Following that report, data on the classes of 1962, 1963, 1964 and 1965 
were obtained in 1968 and 1969 (Brandt, 1969). Similar initial testing 
patterns were found as additional classes (1964 and 1965) were evaluated. 
A second follow-up study of the 1962, 1963, 1964, and 1965 classes was done. 
Several implications for evaluation were drawn from the analysis of those 
studies. It was determined that the process of problem solving and the 
manner in which mental health and public health concepts were taught should 
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be evaluated. Students who progressed from fundamentals of nursing to med- 
ical-surgical nursing attained a higher level of achievement th«n students 
who progressed from fundamentals to maternal and child nursing and then 
to medical-surgical nursing, indicating the need to consider sequencing 
of courses in the revised curriculum. 

Input Evaluation 

The PDK Study Committee indicates that input evaluation includes the 
information needed to make decisions regarding how objectives may be stated 
operationally and the potential of the procedural design in meeting the 
objectives (Stuff lebeam, 1971, p. 223). The second rnd third aims of the 
grant proposal, to formulate a conceptual framework for curriculum change 
and to propose a revised curricular plan for the baccalaureate program 
in nursing, and the fourth aim, to implement the proposed curricular plans, 
refer to decisions resulting from the input evaluation* 

Conceptual framework . During the process of formulating a conceptual 
framework for curricular change, seventeen schools were asked if they 
would share their conceptual frameworks, objectives, philosophy statements, 
statements about learning, testing, research reports on their curriculum 
and course outlines. Materials shared by other schools were made available 
to the faculty. A Models Task Force worked with the Undergraduate Curri- 
culum Study Cortmittee to develop several programs on models and nine 
different theoretical types of curricular organizations were presented in 
a chart with a bibliography. 

In contrast to some schools of nursing, the faculty did not wish to 
organize their curriculum around one major concept or one already developed 
theory. A number of conceptual models were drawn together in the decisions 
resulting In the total conceptual framework. A core content model with a 
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terminal quarter of nursing electives in relation to expanded roles or 
specialized nursing roles was adapted. 

Course development . In order to organise the work on the development 
of core content, faculty members were asked to develop core content 
for the major areas which they had previously delineated. The following 
task for. as, which included student participants were formea; 1) Human 
Development Theory, 2) Interpersonal Interactional Theory, 3) Nursing Process, 
4) Dynamics of Illness, 5) Evolution and Futvve of the Health Care Systems, 
and: Health Care Delivery Process and Patterns, 6) Rerearch anc 1 Scholarship 
Skills, and 7) Social and Cultural Systems and Social Activist's Role in 
Health Care Changes. These task forces were to review ideas already proposed, 
encourage brainstorming to see if further topics, unit and sub-topics 
were needed, review the literature, reviev? all previous course outlines 
and related material for additional ideas, review the philosophy, objectives 
and model already accepted and outline the actual course content and behaviors 
desired in relation to content. After some time and a monumental amount of 
effort, decisions were made regarding courser drawn up. These were reviewed 
by the faculty, modified, reviewed again and accepted by total faculty vote 
In September 1972. Similarly, sequencing of course content was proposed, 
reviewed and modified in order to provide the opportunity for students to 
gain maximized learning experience. 

Assessment of staffing, time, budget requirements and availability 
of facilities for educational experiences are all necessary components of the 
input evaluation. These factors are currently being delineated and nego- 
tiated by appropriate committees, departments, and faculty members within the 
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School of Nursing. Additional use of media for instructional components 

in the revised curriculum is also being evaluated by the faculty. Decisions 

have yet to be made in these areas. 

Test selection for data collection 

In order to provide data for the product evaluation, a plan war- de- 
signed for testing students and faculty in reference to specific charac- 
teristics. Decisions had to be made regarding the tests to be selected 
and the procedures to be followed. An intensive review of the literature 
was conducted in order to assess the relative merits of a variety of in- 
struments measuring attitudes, characteristics, achievement and perceptions 
of college climate. A doctoral dissertation and a master 1 s thesis provided 
some, data on characteristics of our own students, and the selection of 
some of the instruments used in these reports was considered. It was 
decided that Robert Stern 1 s Activity Index, College Characteristics Index, 
and Organizational Climate Index , the Myers-BriERs Tyge I ndicator and 
Shostrom's Personality Orientation Inventor y would be used for psychological 
testing. In addition, a biographical questionnaire was designed. Based 
on the information that over forty percent of the students entering the 
professional part of the program in the sophomore year are transfer stu- 
dents, and because of the difficulties encountered in attempting to identify 
which freshmen students will be in nursing, it was decided to test sophomore 
students as they begin their nursing experience and senior students jusc 
prior to graduation. Testing of sophomore students began in Fall 1972. 
In order to obtain comparable data on the faculty and minimize possible 
testing effects, the decision was made to test faculty with the Stern 
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Indices and a biographical questionnaire in Spring of 1973, to be 
repeated in Spring of 1976, Members of the faculty who are appointed 
after the Spring 1973 testing date will be evaluated at the time of 
their appointment. Difficulties encountered in connection with the process 
of testing provided additional data for the process evaluation ♦ Cne' of 
the areas of concern to the faculty in terms of the product evaluation 
was the collection of data in reference to rating of clinical performance, 
A» review of the literature and consultation v?ith knowledgable nurse 
educators failed to reveal any instrument yet developed to adequately 
measure this dimension. It was decided that such a measure would have to 
be designed. 

In addition co the collection of data mentioned above, it was decided 
that student records would be reviewed and scores collected from the 
Washington Pre-College Entrance Examination, high school grades and previou 
college experiences. Assessment of student achievement was also planned 
through consideration of scores on such tests as Nursing State Boards and 
The National League for Nursing tests. 

Process Evaluation 

Process evaluation involves an assessment of the design and decisions 
made to implement the project. Process evaluation wilt be considered from 
five aspects: 1) evaluation of whether or not the curricular materials 
reflect the desired direction of change; 2) assessment of the testing 
program; 3) evaluation of the time and cost effectiveness of the revised 
curriculum; 4) assessment of the instructional methods selected for use; 
and 5) evaluation of the ourricular process- used to bring about the 
curriculum revision. 
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Curricular materials and direction of change 

Curricular materials of the current and revised curricula will be 

rated by a panel of outside experts in relation to standardized forms and 

stated criteris. As a part of the formative evaluatiow, it if planned 

that content analyses will be used to determine the internal logical* 

consistency among the philosophy, objectives, learning thecr/, content 

outlines and evaluation methods. Curricular materials will also be 

evaluated in terms of external standards answering questions such as the 

f 

following: 1) Are levels of behaviors in the objectives consistent with 
levels of behaviors considered desirable by members of the profession? 
2) Are all major areas of content felt essential by mer.ber of the pro-* 
fession included? 3) Are changes in the direction and amount stated as 
desirable by the conceptual framework, the rationale for change and the 
decisions made % by faculty? and 4) Is there flexibility in the curriculum 
to allow for change without major revision? 

Testing program for student* and faculty 

In order to initiate the program of student evaluation, the proposal 
was reviewed by a behavioral science review board of the University. Since 
students may not be required to take the psychological examinations, the 
problem of maximizing participation was encountered. Although participaion 
was high when class time was allocated for testing, it was consideraily 
reduced when students were asked to volunteer additional time to complete 
the battery. Alternative plans to remedy this difficulty are being con- 
sidered. 
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During the process of collecting data from student records several 
problenis became evident. Transcripts and other sources of data were 
frequently not available in the School of Nursing files* They were found 
to be located in several areas oa campus* In many instances the storage 
system made the process of retrieval extremely difficult. As there *is no 
computerized record-keeping system making data readily available, it was 
decided that the redesigning of the system of record collection and 
storage within the school was indicated. Plans are currently under way 
to facilitate this process. 

Time and cost effectiveness 

Evaluation of the revised curriculum as compared to the current 
curriculum is planned in relation to time and cost factors. With the 
assistance of the business office of the School of Nursing, data will be 
collected on the cost of alternative rotation patterns being suggested, 
cost per ctedit of the major units of the curriculum, average cost per 
student and instructional alternatives which w*!l increase achievement and 
reduce cost. Plans are currently being made to computerize much of the 
data which will help in the evaluation of these factors. 

Instructional methods 

Data will be gathered in relation to the use of types of classroom 
interaction and levels of questioning, and how these factors are related 
to attainment of the objectives. 

Currlcular process 

Through an analysis of committee meeting minutes, data will be 



Wolf 23 

gathered in relation to the following questions: 1) What forces helped 
bring about the change? 2) What forces acted as barriers or modifying 
factors? 3) How were the decisions made? 4) What were the communication 
networks? 5) What were the problems arising from multiple decision makers 
relating to the same decision? 6) What were the problems arising from the 
interdependency of decisions? Administrative reorganization of the school 
is occurring and this is the subject of a separate study. This data may 
assist us in determining additional administrative effects. Although some 
aspects of the process evaluation are already being conducted, this 
stage of evaluation will continue throughout the grant period. 

Product evaluation 

Product evaluation provides the data necessary to assess the extent 
to which objectives are being attained. Evaluation of the achievement of 
students in the current curriculum may be seen as product evaluation. 
This was discussed as input into the context evaluation which assisted 
our school in making a decision to change our curriculum. This inter- 
relationship is illustrated in the model by a feedback line from product 
evaluation into context evaluation. 

The fifth aim of the grant is to evaluate the contribution of the 
revised curriculum. As has been discussed earlier, testing of students 
as they enter and graduate from the revised curriculum is planned. Data 
will continue to be collected from student records. Faculty will be tested. 
Although there will be no opportunity to evaluate the performance of 
graduates from tjfe revised curriculum during the grant period, it is 
hoped that anoU^K grant will .be approved for continuation of the evaluation, 
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Summary 

The first part of this paper presented an overview of some major 
theoretical issues in curriculum evaluation: definitions of evaluation, 
research versus evaluation, models of evaluation, range of the data to 
be included and the number and types of personnel • We have chosen 
Gephart's model as the model to guide the course of *-he evaluation 
program. In the second part of the paper sume factors within our situation 
that have modified the scope and rigor of the evaluation were discussed. 
These included: limited grant funds, time factors in the actual situation, 
school and state budgets, and consultation quests. In the final 
section an overview of the progress and problems in the evaluation 
program is summarized in relation to the parts of the model. 
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